Scene Parsing by Data Driven Cluster Sampling

This paper presents a data-driven cluster sampling framework for parsing scene images into generic regions (such as the sky, mountain and water) and objects (such as cows, horses and cars). We adopt generative models for both generic regions and objects, thus their likelihood probabilities are comparable and are learned under a common information projection principle. The inference algorithm follows the data-driven Markov Chain Monte Carlo (DDMCMC) paradigm where the object and generic region models cooperate and compete for an optimal interpretation of the scene in a Bayesian framework. The algorithm has two phases: (i) Bottom-up computation for generating data-driven proposals. There are two types of proposals: proposals for regular-shape objects using the active basis models and proposals for both generic regions and irregularshape objects (such as crouching cows) by training a set of discriminative models on the appearance. A candidacy graph is constructed to summarize all the bottomup information by treating proposals as nodes and cooperative/competitive contextual relations among proposals as +/edges. (ii) Top-down computation by cluster sampling for seeking the optimal solution that maximizes the Bayesian posterior probability. The cluster sampling algorithm consists of reversible jumps to explore the solution space effectively. At each step, it samples the +/edge probabilities on the candidacy graph and divides the candidacy graph into a set of composQ. Zhou†,∗, T.F. Wu‡,∗, W.-Y. Liu† and S.-C. Zhu‡,?,∗ Department of †Electronic and Information Engineering, Hua Zhong University of Science and Technology, Wuhan, China Department of ‡Statistics and ?Computer Science, University of California, Los Angeles, USA ∗Lotus Hill Research Institute (LHI), Ezhou, China E-mail: [email protected], {tfwu, sczhu}@stat.ucla.edu, [email protected] ite connected-components (CCCP’s) on which the reversible jumps are carried out. In experiments, our algorithm outperforms the state-of-the-art methods on the LHI 15-class dataset and obtains comparable results on the MSRC 21-class dataset (the LHI 15-class dataset has more accurate annotations and we release it with this publication).